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ABSTRACT 

Nonlinear  effects,  or  non-Wiener  behavior,  have  been 
obser\ecI  m  several  adaptive  filtering  applications,  such  as 
adaptive  equalization  of  wideband  communication  signals 
contaminated  by  narrowband  interference,  adaptive 
narrowband  noise  cancellation,  and  adaptive  linear 
prediction  of  narrowband  processes.  The  nonlinear  effects 
have  been  especially  prominent  when  narrowband 
processes  have  been  involved,  and  revealed  b\’ 
performance  better  than  expected  from  the  conventional 
Wiener  filter  for  that  scenario.  Furthermore,  in  the 
adaptive  noise-canceling  scenario,  almost  deterministic 
semi -periodic  dynamic  behavior  of  the  adaptive  filter 
weights  has  been  observed  In  the  adaptive  equalization 
and  prediction  scenarios  the  weight  behavior  is  also 
dv  namic,  albeit  of  a  different  nature.  The  dynamic  weight 
behavior  associated  with  nonlinear  effects  is  shown  to  be 
the  result  of  hov\  the  conventional  NLMS  adaptive  filter 
tracks  the  equivalent  model  for  the  desired  signal. 


U  INTRODUCTION 

Non-linear  effects  have  been  demonstrated  in  adaptive 
noise  canceling  [1,  2],  interference  contaminated  adaptive 
equalization  [1,  3],  and  adaptive  linear  prediction  [4], 
Often  these  nonlinear  effects  arc  strongest  when  involving 
narrowband  processes  and  when  adaptive  filter  stepsizes 
are  relativ  ely  large.  The  nonlinear,  or  non-Wicner,  effects 
are  characterized  bv  performance  better  than  that  of  the 
Wiener  filter  of  the  same  structure,  and  by  dynamic  or 
time-vary  ing  behavior  of  the  adaptive  filter  weights 

As  was  shown  for  the  noise  canceling,  equalizer,  and 
prediction  contexts  [1  -  4],  nonlinear  effects  arise  in  LMS 
as  well  as  NLMS  adaptation  We  concentrate  here  on  use 
of  the  NLMS  algonthm.  The  nonlinear  effects  were 
attributed  to  the  error  signal  feedback,  which  is  used  in  the 
NLMS  weight  update,  and  led  to  the  establishment  of  a 
bound  for  the  performance  under  nonlinear  effects  [2] 

We  show  that  the  mechanism,  by  which  the  error 
feedback  results  in  the  observed  nonlinear  effects,  is 
associated  with  the  instantaneous  information  it  carries 
about  the  deviation  between  the  desired  signal  and  its 
NLMS  modeled  version.  The  structure  for  the  desired 
signal  will  be  related  to  the  underlying  optimal  estimator. 


which  ma\  have  a  structure  that  does  not  fit  the 
conventional  NLMS  tapped  delay  line  model.  When  the 
structure  underlying  the  desired  data  differs  from  the 
NLMS  model  that  is  being  adapted,  the  potential  for  non- 
Wiener  effects  is  created.  We  show  that  NLMS  adaptation, 
by  virtue  of  its  inherent  tracking  capability,  can  often  form 
a  time-vary  ing  tapped  delay  line  model  that  reasonably 
well  approximates  an  underlying,  time-varying  Wiener 
filter  equivalent 


2.  NLMS  ADAPTATION 


The  NLMS  adaptation  algorithm  is  as  follows: 

n  n  n 

e„  =d-d„ 


n  n 


(1) 


To  emphasize  the  situations  under  which  nonlinear  effects 
are  prominent  a  large  stepsize  is  chosen,  /I  =  1 ,  so  that 
adaptation  is  the  fastest.  Furthermore,  it  can  then  be  shown 
that  the  a  posteriori  error  equals  zero  [5],  i.e 


n 


=  0 


(2) 


An  additional  property  of  the  NLMS  algonthm  is  that 
the  a  posteriori  weight  \  ector  is  such  that  the  norm  of  the 
weight  vector  increment  is  minimized,  while  (2)  serves  as 

the  constraint  Note  from  (1)  that  the  change  from  to 
w  IS  bv  necessits  in  the  direction  of  u  „  If  the  desired 
signal  has  the  same  structure  as  that  used  in  the 

modeling  process,  ie  some  fixed 

weight  vector  then  the  NLMS  adaptation  converges 

to  that  weight  vector  (it  produces  a  posteriori  errors  of 
zero  and  weight  vector  increment  norms  of  zero). 

From  (I)  we  sec  that  the  current  estimate  for  the 
desired  signal  depends  on  the  current  weight  v  ector  and 
the  current  input,  while  the  current  weight  vector  depends 


on  the  previous  weight  vector,  the  previous  error,  and  the 
previous  input,  etc.  Assuming  that  we  operate  in  steady 
state,  so  that  the  effects  of  the  initial  weight  vector  choice 
have  vanished,  the  current  estimate  for  the  desired  signal 
is  some  function  of  all  past  desired  signal  samples  and  all 
current  and  past  samples  of  the  input  vector  elements. 

(3) 


The  difl'erence  between  the  conventional  and  two- 
channel  cases  lies  in  the  definition  for  the  NLMS  input 

vector  U^.  For  a  general  two-channel  case,  the  NLMS 
input  vector  can  be  defined  as  follows. 


u 


n 


(8) 


This  NLMS  estimate  is  not  necessarily  optimal 

3.  WIENKR  FILTERS  &  DATA  STRUCTURES 

If  we  assume  that  the  processes  in  (3)  are  jointly  Gaussian 
and  wide-sense  stationar\'  then  the  optimal  (Wiener) 
estimate  for  the  desired  signal  is  time-invariant  and  linear. 

-  X  d„-k  +  X  ) 

A-l  it=0 

If  the  processes  are  such  that  only  a  finite  memor\'  depth  is 
required,  say  of  I.  and  M  samples  respectively,  then  the 
optimal  estimator  takes  on  the  following  structure. 

^  /.  M-\ 

^L.op,  =  E 

A -I  A  O 


The  vectors  and  are  referred  to  as  the  auxiliarv  and 

the  reference  channel  respectively  The  conventional 
NLMS  algorithm  uses  only  the  so-called  reference 

channel,  i.e.  In  order  to  match  the  structure  of 

the  optimal  estimator  in  (6),  the  NLMS  input  would 
contain  an  au.xilian  channel  =d„  The  auxiliarv 

channel  definition  depends  on  the  application 

The  corresponding  Wiener  filter  (WF)  designs  follow 
from  the  Wiener-Hopf  equation. 


The  performance  of  tlie  resulting  WF,  using  /.  auxiliar> 
taps  and  Af  reference  taps,  is  given  b\ 


We  note  that  the  latter  can  be  rewritten  as  follows; 

^  n, opt 


fh''" 

h“''l 

L"  opt 

■■  opt  J 

- 

(6) 


The  consequence  of  this  development  is  that  we  now 
have  an  underlying  structure  for  the  desired  data  itself 


^ n  ^njipt  ^ n. opt 

=  Kpl^n  +^n..,pl 


(7) 


This  structure  shows  that  the  desired  data  can  be 
represented  with  a  two-channel  L.Tl  model,  and  that  the 
corresponding  representation  error  equals  the  two-channel 
Wiener  filter  error. 

The  model  inherent  in  the  NLMS  algorithm  in  (1 )  is  of 
the  form  of  the  structure  in  (6).  The  distinction  lies  in 

whether  or  is  used.  The  difference  resides  in  the 
importance  of  the  past  of  the  desired  signal  samples  If 
there  is  no  information  m  {d „  *  about  ,  then 

effectively  is  the  same  as  ,  and  NLMS  -  that  is  to 
say  -  converges  to  (a  neighborhood  oO  NLMS 
then  approaches  the  Wiener  filter  in  performance. 


(10) 


The  nonlinear  effect  scenarios  usual  1\  corresponded  to 
(jomtK  )  wi de-sense  stationar)  processes,  m  which  case 
the  resulting  WF  solutions  are  linear  time-invanant  (LTl) 
filters.  The  two-channel  Wiener  filter  design  and 
performance  evaluation  are  based  on  auto-  and  cross- 
correlation  information,  and  can  be  done  efficientK  [6,  7] 


4.  APPLIC  ATION  SC  ENARIOS 


For  each  application  scenario,  the  two-channel  version  is 
reflected  in  the  figures  below  For  each  of  the  input 
channels  a  tapped  delay  line  exists  internal  to  the  NLMS 
structure.  Conventional  NLMS  does  not  use  ,  i.e.,  the 

auxiliary  channel  partition  of  the  input  is  inacti\  e. 

The  adaptive  noise-canceling  (ANC)  scenario  is 
reflected  in  Fig.  1.  Its  aim  is  to  predict  a  desired  signal 
from  a  reference  input,  which  presumabK  contains 
information  about  the  desired  signal  because  it  is 
correlated  therewith.  In  many  cases  performance  is 
increased  when  the  desired  signal  is  estimated  from  its 
own  immediate  past,  rather  than  from  a  different  signal 
The  choice  for  the  auxiliary'  signal  m  ANC  is  therefore  the 

immediate  past  of  the  desired  signal,  i.e.  X^  =  d  ^  , 


The  interference  contaminated  adaptive  equalization 
(AEQ)  scenario,  in  training  mode,  is  reflected  in  Fig.  2. 


Nonlinear  effects  were  reported  for  scenarios  where  the 
interference  was  strong  relative  to  the  signal  ,  which 

itself  was  strong  relative  to  the  noise  .  In  this  case  we 

choose  p .  the  interference  at  the  center  tap  where  the 

signal  estimate  is  desired,  as  the  au\iliar\  channel. 

The  adaptive  linear  prediction  (ALP)  scenario  is 
rellected  in  Fig.  3.  In  this  case  we  are  interested  in 
predicting  the  desired  signal  from  its  far  past,  i.e.  A  and 
more  samples  in  the  past.  Generally  a  signal  can  be 
predicted  much  better  from  its  recent  past  than  from  its  far 
past,  so  that  the  immediate  past  is  chosen  here  as  the 
auxilian*  channel 

From  the  Wiener  filtering  point  of  view,  the  reference 
signal  is  related  to  the  desired  signal  in  each  of  the  above 
cases.  In  ANC  we  assume  there  is  statistical  correlation 
between  the  two,  in  AEQ  the  desired  signal  is  a 
component  of  the  reference  signal,  and  in  ALP  the 
reference  signal  is  the  far  past  of  the  desired  signal. 


5.  TIME- VARYING  WIENER  FILTERS 

Strong  nonlinear  effects  have  been  observed  for 
narrowband  processes,  which  are  modeled  well  with  an 
AR(1)  model.  For  these  processes,  together  with  high 
signal  to  noise  ratios,  the  Wiener  filter  of  (6)  -  for 
increasing  L  and  M  ver\'  quickly  saturates  to  the  Wiener 
filter  of  (4).  The  latter  happens  for  L  =  l  in  ANC,  ALP 
and  AEQ  [8]  Consequently,  the  desired  signal  has  an 
underl>ing  structure  as  giv  en  in  (7),  where  -  furthermore 
the  dimension  of  the  auxiliary’  channel  vector  is  one.  The 
corresponding  data  structure  can  be  written  as  follows. 


h'',u 

op{  n 

n,opi 

hr 

i-:;] 

V^opl 

We  use  the  concept  of  linking  sequences  [8], 

r  n-i 

V, 

to  write  the  data  structure  in  ( 1 1 )  in  equivalent  form 

dn  =  ^  ^  n,op, 


(12) 


(13) 


+  f 


n.opl 


The  vector  has  a  1  in  the  (/+1)'^  position  as  its  only 

non-zero  element.  The  auxiliary  signal  can  be  linked  with 
any  of  the  elements  in  the  reference  vector,  so  that  the 
following  affine  combination  is  the  most  general 
equiv  alent  data  structure. 


rf.  = 


la, I," +!> 


r  +  £* 


n,opl 


(14) 


The  bracketed  term  represents  Wiener  filter  coefficients 
operating  on  the  reference  vector  onl\’,  while  still  incurring 
the  error  of  the  optimal  two-channel  Wiener  filter.  This 
reference-only  Wiener  filter  equivalent  has  a  time-var\ing 
component,  due  to  the  presence  of  the  linking  sequences, 
in  addition  to  a  time-invanant  component,  corresponding 
to  the  reference  partition  of  the  two-channel  WF  weights. 

I’he  model  inherent  in  conventional  NLMS,  as  in  (1), 
is  of  the  same  form  as  the  first  term  in  (14).  The  latter 
affine  combination  represents  the  manifold  of  weight 
vector  solutions  from  which  NLMS  takes  its  a  posteriori 
weight  vector,  while  minimizing  the  norm  of  the  change  in 
the  weight  vector.  If  the  NLMS  weights  could  capture  the 
behavior  of  the  bracketed  term  in  (14),  NLMS 
performance  would  equal  that  of  the  two-channel 
(optimal)  Wiener  filter.  In  practice,  NLMS  can  capture  a 
time-\ar\ing  weight  vector  a  posteriori  at  best,  so  that 
NLMS  a  priori  performance  is  reduced  -  relative  to  the 
optimal  two-channel  WF  performance  -  b\  tracking  error 
as  well  as  estimation  error.  Nevertheless,  in  many 
scenarios,  conventional  NLMS  may  be  able  to  track  to  the 
e.xtent  that  its  performance  is  better  than  that  of  the 
corresponding  conventional  Wiener  filter. 

6.  DYNAMIC  WEIGHT  BEHAVIORS 


For  high  SNR  conditions,  and  in  scenarios  as  indicated  in 
the  previous  section,  the  bracketed  term  in  (14)  forms  the 
target  weight  vector  for  NLMS.  This  implies  that  the 
weight  behavior  of  NLMS,  in  particular  for  a  stepsize 
jU  =  1 ,  follows  the  behavior  indicated  above.  This 
beha\  ior  depends  on  the  linking  sequences,  which  in  turn 
depend  on  the  particular  application  scenarios. 

We  now  illustrate  the  dy  namic  weight  beha\  ior  for  the 
ANC  scenario  with  narrowband  AR(1)  signals  (with 
desired  and  reference  comple.x  poles  at  and 

respectively),  for  (/v,M)  =  (l,2)  For  the  bracketed 
term  in  (14)  we  then  find  [8] 


h 


77 

n 


-  h’"  o'*”* 

'‘oplFn-l 


+  Kr. 


(15) 


where  the  and  7]^  terms  arc  results  due  to  the  driving 

noises  of  the  corresponding  AR(1)  processes.  For  the 
remaining  linking  sequence  the  following  holds: 


p'"'!  =  pjp:'pZ'',  +  < 


\p 


''"-"'Ip;':'; +< 


(16) 


Apart  from  the  stochastic  terms  due  to  AR(1)  driving 
noises  and  the  time-invariant  term  corresponding  to  the 


reference  partition  of  the  optimal  two-channel  WF,  the 
weight  vector  target  for  NLMS  is  seen  to  consist  of  a  time- 
var>  ing  term  that  rotates  from  sample  to  sample  by  the 
difference  of  the  center  frequencies  of  the  desired  and 
reference  processes.  Further  note  from  (15)  that  the  second 
weight  \ector  element  is  rotated  with  respect  to  the  first 
weight  vector  element  by  the  complex  conjugate  of  the 
reference  pole. 

To  illustrate  the  dvnamic  NLMS  weight  behavior,  for 
the  ANC  case,  let  S/VR=80dB.  p,,  =  0.99e"”*  . 

and  =  0.99t?^*''^^  Fig.  4  shows  the  absolute  error  in 
dB,  during  iterations  4700-5000,  which  is  in  steady  state. 


4700  4750  4800  4850  4900  4950 


Iteration  Number 

Fig.  4.  Steady-state  NLMS  (light)  and  WF  (dark)  error 

We  observe  that  NLMS  outperforms  the  LTl  WF, 
almost  on  a  sample-for-sample  basis.  Over  the  steadv  -state 
interv  al,  WF(0,2)  the  implementation  of  the  optimal  WF 
-  realizes  MSB  of  15  82  dB,  while  NLMS(0.2)  realizes 
MSE  of  12.04  dB.  The  theoretical  min  MSE  is  16.92  dB; 
WF(0,2)  is  about  1  dB  less  because  the  signal  power  in 
this  realization  is  about  1  dB  lower  than  its  expected  value. 

Fig.  5  shows  the  dv  namic  w  eight  behav  ior  of  NLMS 


Iteration  index 


Fig.  5.  NLMS  and  WF  (constant)  weight  behavior. 


The  behavior  of  the  imaginar\  part  is  similar  We  observ  e 
that  the  weight  behavior  is  semi-periodic,  with  a  period  of 

10  as  expected  from  (16)  when  —0.2k  .  We 

also  observe  that  the  second  weight  element  is  shifted 
relative  to  the  first  one  b\  Q.5k  ,  corresponding  to  0)^ . 

Comparing  Figs  4  and  5  we  observe  that  the  dynamic 
weight  behavior  is  strongly  periodic  when  signal  levels  are 
relatively  high  (iterations  4800-4819),  and  less  so  when 
signal  levels  are  lower  (iterations  4700-4719).  When 
signal  levels  are  high,  the  driving  noise  related  terms  in 
(16)  are  less  influential  and  therefore  periodicit)  is  more 
pronounced  and  more  clearly  centered  about  the  reference 
partition  of  the  WF(1,2)  filter.  The  latter  is  further 
illustrated  by  the  dynamic  weight  behavior  of  NLMS(0,2) 
during  iterations  4700-4719,  in  Fig.  6,  and  during 
iterations  4800-4819,  in  Fig.  7. 


Fig.  7.  NLMS  and  WF  weights  during  4800-4819. 


Recall  that  NLMS  is  of  necessity  one  step  behind  in 
tracking  its  time-varving  equivalent  target.  Figs.  6  and  7 
illustrate  that  the  weight  v  ector  one  sample  off  is  general  1> 
still  closer  to  the  target  than  the  WF( 0,2)  weights  are. 


6.  SUMMARY 

We  have  illustrated  that  a  conventional  NLMS  adaptive 
filter,  using  a  tapped  delav  line  of  reference  values,  can 
have  a  time-varying  Wiener  filter  equivalent  as  its  target. 
As  a  result  of  its  tracking  ability  ,  the  conventional  NLMS 
adaptive  filter  can  achieve  better  performance  than  the 
corresponding  conv  entional  Wiener  filter  for  a  given  vvide- 
sense  stationary  scenario.  A  noise-canceling  example 
shows  semi-periodic  weight  dynamics  in  tune  with  the 
behav  ior  of  the  time-vary  ing  Wiener  filter  equiv  alent. 
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